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DETAILED ACTION 
Response to Arguments 

1. Applicant's arguments on page 1 1, first paragraph, filed April 19, 2004, with respect to 
claims 39 and 44-46 have been fully considered and are persuasive. The objection and 35 U.S.C. 
§ 1 12, second paragraph rejection of claims 39 and 44-46 have been withdrawn. 

2. Applicant's arguments filed April 19, 2004 have been fully considered but they are not 
persuasive. 

3. On page 12, second paragraph, applicant states, "In reading claims 1 and 31 on 
Ratakonda, the Examiner has taken Ratakonda's teachings out of order." The examiner 
respectfully disagrees. Claim 1; line 1-3 states, "A method for summarizing a content of an input 
video sequence . . . said method comprising", claim 3 1 being similarly worded. The transitional 
term "comprising", which is synonymous with "including," "containing," or "characterized by," 
is inclusive or open-ended and does not exclude additional, unrecited elements or method steps. 
See, e.g., Genentech, Inc. v. Chiron Corp., 1 12 F.3d 495, 501, 42 USPQ2d 1608, 1613 (Fed. Cir. 
1997) ("Comprising" is a term of art used in claim language which means that the named 
elements are essential, but other elements may be added and still form a construct within the 
scope of the claim.); Moleculon Research Corp. v. CBS, Inc., 793 F.2d 1261, 229 USPQ 805 
(Fed. Cir. 1986); In re Baxter, 656 F.2d 679, 686, 210 USPQ 795, 803 (CCPA 1981); Ex parte 
Davis, 80 USPQ 448, 450 (Bd. App. 1948) ("comprising" leaves "the claim open for the 
inclusion of unspecified ingredients even in major amounts"). See MPEP § 21 1 1.03 Transitional 
Phrases. Additionally, the claim does not provide a limitation that states that the method steps 
(or the program steps in claim 3 1) are necessarily performed in order. Even though step (b) 
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computes a "content value in said selected frame cluster," it is still believed that steps (a) and (b) 
could be performed in a different order or, for example, simultaneously. Note that step (b) states 
that a content value is computed in said selected frame cluster, not necessarily for the whole 
frame cluster, which could include any number of content values such as histograms or action 
measures for each individual frame and could, conceivably be calculated before the frame cluster 
is selected. Finally, the term "content value" is not further defined in the claims and has thus 
been given the broadest, most reasonable interpretation. Although the claims are interpreted in 
light of the specification, limitations from the specification are not read into the claims. See In 
re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The examiner believes that 
Ratakonda' s action measure satisfies this limitation. 

4. On page 13, second paragraph, applicant states, "Applicants do not understand the 
Examiner's reading of the last item (c) in claims 1 and 31 on col. 7, lines 25-33 on Ratakonda, 
nor do Applicant's understand how the portion of Ratakonda at col. 10, lines 35-42 corresponds 
to that portion at col. 7. Item (c) in claims 1 and 3 1 recites the use of a computed content value 
from (b) to cluster remaining frames in the input video sequence." Ratakonda discloses a 
minimization procedure to find keyframes in a distribution that varies adaptively to the amount 
of "action" in the shot (column 7, line 25-33). The first keyframe represents the video shot 
having the lowest cumulative action measure (content value) C (Figure 6\ column 7, line 10-33). 
Keyframes are then clustered based on similarity (column 9, line 40-56) and selected within 
subsequent clusters according to the "largest difference from the previous keyframe criterion," 
expressed in terms of the action measure (column 10, line 35-42). The examiner believes that 
these sections read on the claimed limitations (additional sections cited for clarity). 
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5. On page 14, second paragraph, applicant states, "Lim does not teach or even remotely 
suggest the application of singular value decomposition in the context claimed in claims 17 and 
47 (or claims 6-9 and 36-39, for that matter)." The examiner respectfully disagrees. In response 
to applicant's argument that Lim is nonanalogous art, it has been held that a prior art reference 
must either be in the field of applicant's endeavor or, if not, then be reasonably pertinent to the 
particular problem with which the applicant was concerned, in order to be relied upon as a basis 
for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. 
Cir. 1992). In this case, Lim discloses the use of singular value decomposition to reduce the 
dimensionality of a spatial aggregation map representing the visual-content signature for the 
visual document (column 9, line 14-40). Lim discloses, "Embodiments of the invention provide 
a system for creating visual "keywords" from visual documents. The visual keywords are 
prototypical, visual entities present in the content of a given distribution of visual documents, 
which can include digital images and video-shot keyframes " (emphasis added; column 6, line 18- 
30). The examiner believes that at least this statement puts the document in the proper context of 
the instant application. 

6. Finally, since the applicant has not set forth arguments for the remainder of the rejections, 
namely claims 10 and 40 over Ratakonda in view of Uchihachi, and claims 13-16 and 43-46 over 
Ratakonda in view of Castelli, it has been assumed that applicant agrees with these grounds. 

Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1 (a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1 (2) of such treaty in the English language. 

8. Claims 1-5, 1 1, 12, 31-35, 41, and 42 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Ratakonda (USPN 5,956,026, previously cited). 

9. Regarding claims 1 and 31, Ratakonda discloses a method and computer-readable 
medium containing a program for summarizing a content of an input video sequence, said input 
video sequence comprising a plurality of frames (column 3, line 21-29), said plurality of frames 
being grouped into a plurality of video segments (column 4, line 42-63), said method and 
program comprising: (a) selecting a frame cluster (represented by keyframe kl) in said input 
video sequence which corresponds to a most static one (minimized action) of said video 
segments (column 6, line 45-60; Figure 6; column 7, line 10-33; column 8, line 31-63); (b) 
computing a content value (cumulative action measure C) in said selected frame cluster (video 
shot) (column 6, line 26-44); (c) using said computed content value to cluster remaining frames 
(keyframes representing video shots) in said input video sequence (column 7, line 25-33; column 

9. line 40-56; column 10, line 35-42). 

10. Regarding claims 2 and 32, Ratakonda discloses that in said step (a), said frame cluster is 
selected using a refined feature space representation (treat each histogram as a feature vector of 
its associated frame) of said input video sequence (column 9, line 44-56). 

1 1 . Regarding claims 3 and 33, Ratakonda discloses that in said step (a), each of said 
plurality of frames is transformed into a histogram vector indicative of a spatial distribution of 
colors in said each of said plurality of frames (column 4, line 48-54). 
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12. Regarding claims 4 and 34, Ratakonda discloses that in said (a) each of said plurality of 
frames (finest level keyframes) is divided into a plurality of blocks,, each of said plurality of 
blocks being represented by a histogram in a color space indicative of a distribution of colors 
within each of said plurality of blocks (column 1 1, line 1-17; column 14, line 15-47). 

13. Regarding claims 5 and 35, Ratakonda discloses that each of said plurality of frames 
(finest level keyframes) is divided into a plurality of blocks and each said histogram vector 
comprises a plurality of histograms in a color space, each of said plurality of histograms 
corresponding to one of said plurality of blocks (column 11, line 1-17; column 14, line 15-47). 

14. Regarding claims 1 1 and 41, Ratakonda discloses a step (d) including outputting a 
plurality of keyframes, each of said plurality of key frames representative of said clustered 
frames (column 4, line 54-63). 

15. Regarding claims 12 and 42, Ratakonda discloses that said selecting comprises locating a 
cluster (having a histogram vector) closest to an origin (centroid or mean histogram assigned as a 
representative vector) of said refined feature space (column 10, line 5-42). 

Claim Rejections - 35 USC §103 

16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

17. Claims 17 and 47 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ratakonda in view of Lim (USPN 6,574,378, previously cited). 
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18. Regarding claims 17 and 47, Ratakonda discloses a method for summarizing a content of 
an input video sequence (column 3, line 21-29), said method comprising: (a) selecting frames 
(keyframes) from said input video sequence (column 4, line 54-63), said selected frames being 
taken at a fixed (equispaced) interval (column 7, line 42-49); (b) Lim discloses creating a feature 
frame matrix using said selected frames (SAM vectors of visual documents); (c) performing a 
singular value decomposition on said feature frame matrix to obtain a matrix representing said 
video sequence in a refined feature space (see above discussion of claims 1 and 31); (d) 
Ratakonda discloses selecting a cluster (represented by keyframe kl) in said refined feature 
space corresponding to a most static (minimized action) video segment (Figure 6; column 6, line 
55-60; column 7, line 25-33); (e) computing a content value (cumulative action measure C) 
corresponding to said selected cluster (column 6, line 26-44); (f) using said computed content 
value to cluster frames in said input video sequence (column 7, line 25-33; column 10, line 35- 
42). 

19. Claims 6-9 and 36-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ratakonda as applied to claims 2 and 32 above, and further in view of Lim. 

20. Regarding claims 6 and 36, Ratakonda does not disclose that said refined feature space 
representation is obtained using a singular value decomposition of said input video sequence. 
Lim discloses a method and apparatus for indexing and retrieving images using visual keywords 
wherein during indexing (or retrieval) of a visual document, a spatial aggregation map (SAM) of 
occurrences of visual tokens is created which represents a visual-content signature for the visual 
document (column 9, line 14-28). Subsequently, the SAM is input to a singular- value- 
decomposition (SVD) based coding module to produce a refined feature (reduced 
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dimensionality) space representing a coded description of a visual document (column 9 5 line 29- 
37). It would have been obvious to one of ordinary skill in the art at the time the invention was 
made to obtain a refined feature space using a singular value decomposition of an input video 
sequence as taught by Lim in order to reduce the dimensionality and possibly the noise in the 
spatial aggregation map to produce a coded description of a visual document (column 9, line 29- 
33). 

21 . Regarding claims 7 and 37, Ratakonda discloses that minimization (to a refined feature 
space) is performed using frames (keyframes) selected with a fixed interval (equispaced) from 
said input video sequence (column 7, line 42-49). 

22. Regarding claims 8 and 38, Lim discloses that said selected frames (linearized SAM 
vectors) are arranged into a feature frame matrix (X), and wherein said singular value 
decomposition is performed on said feature frame matrix (column 9, line 33-44). 

23. Regarding claims 9 and 39, Lim discloses that said singular value decomposition 
produces a matrix (X), each column of said matrix representing a frame (SAM vectors of visual 
documents) in a refined feature (reduced dimensionality) space corresponding to a frame in said 
input video sequence (column 7, line 33-44). 



24. Claims 10 and 40 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ratakonda as applied to claims 1 and 3 1 above, and further in view of Uchihachi et al. (USPN 
6,535,639, previously cited), hereafter Uchichachi. 

25. Regarding claims 10 and 40, Ratakonda does not disclose a step (d) using said clustered 
frames to output a motion video representative of a summary of said input video sequence. 
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Uchihachi discloses automatic video summarization using a measure of shot importance and a 
frame-packing method wherein a video may be modified during playback to emphasize the 
measure of shot importance in order to de-emphasize less important shots, or skip shots entirely 
that are less important than a predetermined threshold, resulting in a summary or "video skim" 
(column 9, line 53-60). It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to output a motion video representative of a summary of an input video 
sequence as taught by Uchihachi in order to emphasize the measure of shot importance by de- 
emphasizing or skipping less important shots and also allowing the user to generate a personal 
video summary (column 9, line 53-60). 

26. Claims 13-16 and 43-46 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ratakonda as applied to claims 2 and 32 above, and further in view of Castelli et al. (USPN 
6,122,628, newly cited), hereafter Castelli. 

27. Regarding claims 13 and 43, Ratakonda discloses that said step (c) comprises: (c)(1) 
sorting a plurality of vectors in said refined feature space according to a distance of each of said 
vectors to an origin (representative vector) of said refined feature space representation (column 
9, line 49-column 10, line 20); (c)(2) selecting a vector among said sorted vectors which is 
closest to an origin (representative vector) of said refined feature space representation and 
including said selected vector into a first cluster (column 10, line 35-37); (c)(3) clustering said 
plurality of sorted vectors in said refined feature into a plurality of clusters according to a 
distance between each of said plurality of sorted vectors and vectors in each of said plurality of 
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clusters and an amount of information in each of said plurality of clusters (column 10, line 37- 
42). 

28. Ratakonda does not explicitly disclose that the vectors are sorted in ascending order. 
Castelli discloses multidimensional data clustering and dimension reduction for indexing and 
searching wherein eigenvalues (characteristic values of a transformation matrix produced by 
singular value decomposition; column 1 1, line 9-16) are sorted by decreasing magnitude and a 
subset of ordered eigenvalues containing the largest eigenvalues are selected according to a 
selection criterion (column 1 1 , line 25-3 1). Castelli sorts in decreasing order, but it would have 
been obvious to sort in ascending order, depending on the needs of the system. It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to sort a 
plurality of vectors in said refined feature space in ascending order as taught by Castelli in order 
to aid in the selection of the vectors (eigenvalues) according to a user-specified selection 
criterion (column 1 1, line 28-40). 

29. Regarding claims 14 and 44, Ratakonda discloses that in said step (c)(3) said plurality of 
sorted vectors are clustered into said plurality of clusters such that said amount of information 
(action) in each of said plurality of clusters does not exceed an amount of information (action) in 
said first cluster (column 6, line 45-60). 

30. Regarding claims 15 and 45, Ratakonda discloses that said first cluster is composed of 
frames (keyframes) based on a distance variation between said frames (column 8, line 52-63) and 
an average distance between frames in said first cluster (same as the distance since the keyframes 
are equispaced; column 7, line 45-49). 
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3 1 . Regarding claims 16 and 46, Ratakonda discloses that each of said plurality of clusters is 
composed of frames (keyframes) based on a distance variation between said frames (column 8, 
line 52-63) and an average distance between frames in said each of said plurality of clusters 
(same as the distance since the keyframes are equispaced; column 7, line 45-49). 

Conclusion 

32. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. USPN 6,195,458 to Warnick et al. discloses a method for content-based temporal 
segmentation of video. USPN 6,549,643 to Toklu et al. discloses a system and method for 
selecting key-frames of video data to generate a content-based visual summary of video. USPN 
6,751,776 to Gong discloses a method and apparatus for personalized multimedia summarization 
based upon user specified theme. 

33. THUS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Ryan J. Hesseltine 
June 23, 2004 
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